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Consider  an  environment  having  d  possible  states,  where  the  state  of 
the  environment  evolves  throu^i  time  according  to  a  stationary  Markov  chain* 

A  natural  model  for  noise  in  such  an  environment  is  to  assume  that  the 
disturbance  is  driven  by  a  white  noise  process  that  depends  on  the  current 
state  of  the  environment*  In  this  note,  it  is  shown  that  such  a  noise  process 
may  be  represented  by  a  ( d+1 )th  order  ARMA  model* 
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Consider  an  environment  having  d  possible  states,  where  the  state  of 
the  environment  evolves  through  time  according  to  a  stationary  Markov  chain. 

A  natural  model  for  noise  in  such  an  environment  is  to  asras  that  the  dis¬ 
turbance  is  driven  by  a  white  noise  process  that  depends  on  the  current  state 
of  the  environment,  in  this  note,  it  is  shown  that  such  a  noise  process  may 
be  represented  by  a  (d+l)th  order  ARNA  model. 
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SIGNIFICANCE  AMD  EXPLANATION 


Autoregressive  Moving  average  (ANNA)  Models  are  frequently  used  to 
describe  disturbances  associated  with  signals*  Usually,  such  processes  are 

discussed  in  the  context  of  finite  diMsnsional  linear  systeMs.  In  this  note, 

tkt. 

jgft' shov^that  ANNA  Models  also  occur  in  a  rather  different  setting,  naaely  as 
descriptions  of  white  noise  in  a  randomly  varying  environMent.  Such  a  result 
is  useful  in  better  understanding  the  proper  role  of  ANNA  processes  in  systems 


theory. 
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XM  A  MARKOVIAN  ENVIRONMENT 
Peter  N.  Glynn 

1*  Introduction 

Autoregressive  moving  average  (ANNA)  Models  are  frequently  used  to 
describe  disturbances  associated  with  signals.  Usually,  such  processes  are 
discussed  In  the  context  of  finite  dimensional  linear  systems.  In  this  note, 
we  shall  show  that  ANNA  models  also  occur  In  a  rather  different  setting, 
namely  as  descriptions  of  whits  noise  in  a  randomly  varying  environment.  Such 
a  result  Is  useful  in  better  understanding  the  proper  role  of  ARMA  processes 
in  systems  theory. 

|  To  be  precise,  we  consider  an  environment  possessing  d  possible  states, 

lebelled  1  to  d.  The  state  of  the  environment  at  time  n  is  given  by  a 

stationary  Markov  chain  {x_  i  — •  <  n  <  •)  having  an  aperiodic  Irreducible 

transition  matrix  P.  Associated  with  each  state  1  is  a  sequence  of 

independent  identically  distributed  finite  variance  random  variables 

{z  (i)  t  *•<»<•)!  the  sequences  are  independent  of  one  another  and  {X  }. 
n  n 

nolM  process  {Y  i  <  n  <  «•}  is  then  dsfinsd  by  the  nils  Y  * 
n  n 

Z_(X  ).  In  other  words,  Y_  is  driven  by  the  white  noise  Z_(l)  whenever 

n  n  n  II 

Xn  equals  i . 

As  an  application,  consider  the  following  simple  model  for  signal 
disturbances  caused  by  atmospheric  distortion.  Assume  that  the  atmosphere  has 
a  finite  number  of  states  (e.g.  high  and  low  humidity)  and  that  state 
transitions  occur  according  to  a  Markov  chain.  If  the  disturbance  is  assusMd 
to  be  a  white  noise  process  that  depends  on  current  atmospheric  conditions, 
then  the  noise  can  be  realised  as  a  special  case  of  the  above  model. 

Sponsored  by  the  United  States  Army  under  Contract  Mo.  DAAQ29-B0-C-0041. 
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2.  Calculation  of  tho  Spectral  Density 


Ws  first  review  iou  basic  properties  of  the  transition  matrix  P  (see 

Chapter  4  of  [1 1 )  •  The  matrix  P  has  a  unique  d  x  1  stationary  vector  * 

solving  w'P  •  «'  (y'  denotes  the  transpose  of  y).  If  II  is  the  d  x  d 

2 

matrix  with  each  row  identical  to  » 1 ,  then  IIP  “  PH  “  II  ■  II  and  P  -  II 
has  spectral  radius  less  than  1. 

2 

Turning  to  the  calculation  of  the  spectral  density,  1st  p  and  o  be 

d  x  1  vectors  with  i'th  component  given  by  u(i)  “  EZ  (i)  and 

n 

2 

0  (i)  ■  var  Z  (i),  respectively s  Then, 
n 

BY  -  l  E{Y  |  X  -  i}P{X  -  i)  -  If  *|l 
n  ^  n  n  n 

and 

var  Y  -  l  v.EZ2(i>  -  (£  w.EZ  (i))2 
n  *  x  n  j  x  n 

ii  * 

2 

•  W  'O  ♦  u  lT(I-H)w 

where  T  is  a  diagonal  matrix  with  Furthermore,  for  m  >  1 

cov(Yn,Yn+jn)  -  |i*T(Pm  -  It)y  . 

The  spectral  density  of  (Yn)  is  given  by 


f(M  -  57  I  .llkoo»(T  .Y  > 

-  r—  {w*o2  +  u'Td-H  ♦  OCX)  +  D(-X))p> 


(1) 


where  D(A)  ■  \  s^Np*  -  II), 

k-1 

Lemma  1 .  The  inverse  matrix  P(A)  •  (II  -  P  ♦  s^l)  1  exists  for  all  A,  and 
D(A)  -  PP(-A)  -  siXn. 


2 


Proof*  Observe  that  D(A)  *  [  eXXk(P  *  H)*1.  8inc«  P  -  II  has  spactral 

k-1 

radios  lass  than  1,  it  follows  that  tha  Matrix  sob  for  D(A)  converges 


absolutely,  and  hence  D(A)  exists,  hatting  n  ♦  •  in  the  identity 

?  iAk.-k  *ww  n  .  -iX . 

I  e  (P  -  H)(H  -  P  ♦  e  I) 

k-1 

_  _  iXn,_  _%n+1 

-  P  -  n  -  e  (P-B) 


yields 


so  that 


D(X)(n  -  P  ♦  e  I)  -  P  -  n 


eiX(D(A)  ♦  X)(I  -  P  ♦  e”iXI)  -  I 


and  thus  P(-A)  exists.  It  is  easily  seen  that  RP(-X)  -  e  H,  front  which 
D(A)  -  PF(-A)  -  eXXI  follows  frost  (2). 

As  an  isMediate  oonsequence  of  (1)  and  bemma  1,  we  obtain  a  closed  font 
for  f(X). 

Theorem  1.  The  spectral  density  f(X)  of  {Y  }  is  given  by 

n 

f (X)  -  -J-  {y'o2  +  u'T{I-n  ♦  PP(X)  +  PP(-X)  -  Beix  -  ne"iX)u>  .  (3) 

2V 

By  Cramer's  rule  for  calculation  of  matrix  inverses,  g(eiX)P(X)  is  a 
matrix  of  polynomials  in  eXX  of  degree  at  most  r  <  d,  where  g(e^ )  is 
the  d'th  order  polynosdal  det(eiX  -  (P-II ) ) .  since  f(X)  is  real,  even,  and 
non-negative,  it  follows  from  (3)  that  f(X)  may  be  written  as 

..«<«*>, *,i  ?  b..ix<*^*,t*  .  (.I 

**  k-0  *  i-0 

where  a^  -  bQ  -  1,  the  i^'a  and  b^'i  are  real,  and  q  <  p+1  <  d+1 j  here 
the  numerator  polynomial  has  no  roots  outside  tha  unit  disc,  the  denominator 
has  all  roots  inside  the  unit  disc  (recall  that  g(eXX)  t  0  for  all  X),  and 
tha  two  polynomials  have  no  roots  in  cession. 
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3.  ARMA  Representations  for  Yn 

Our  main  result  is  the  following* 

Theorem  2*  There  exists  an  orthogonal  mean  zero  stationary  sequence 

{e  :  -«*  <  n  <  •}  and  a  constant  0  such  that 
n 

Y  +  b*Y  .  ^  b  Y  “6  ^  a«e  .  a  £  _  ^  0  • 

n  1  n-1  p  n-p  n  1  n-1  q  n-q 

A  A  A  A 

Proof.  Let  W_  «  Y  +  b.Y  _+...+  b  Y  (Y  AY  -  EY  )  end  observe  thet 

n  n  1  n- 1  p  n-p  n  *  n  n 

the  spectral  density  of  WR  is  given  by  <*2|  ?  a  eA*(q  *5|2/2*  (see  (4)). 

k-0 

The  spectral  representation  theorem  for  the  weakly  stationary  sequence  Wn 
can  be  utilized  to  write  Wn  as 

n  n  1  n-1  q  n-q 

where  {e^  :  -*•  <  n  <  •}  is  an  orthogonal  weakly  stationary  sequence  (see 
[2],  p*  504,  for  details)* 

It  is  natural  to  ask  whether  all  ARMA  processes  may  be  regarded  as  white 
noise  in  a  Markovian  environment*  However,  it  is  clear ,  from  (4),  that  if  the 
order  of  the  moving  average  component  is  two  or  more  than  the  order  of  the 
autoregressive  part,  that  such  a  representation  is  impossible* 


4.  Martingale  Difference  Representations  for  Yfl 
2 

If  a  *  0,  then  Y  *  y(X  )  is  a  d  state  Markov  chain*  In  the 

n  n 

case  d  ■  2,  the  ARMA  representation  may  be  calculated  explicitly*  First 

observe  that  since  P  is  irreducible,  the  eigenvalue  1  has  algebraic 

multiplicity  1;  since  P  is  aperiodic,  1  is  the  only  eigenvalue  of  unit 

modulus  (see  [3],  p*  536-551).  Thus  the  second  eigenvalue  X  of  P  for 

d  *  2  has  modulus  less  than  unity  and  is  clearly  real*  We  claim  that 

c  *  Y  -  XY  -  (l-X)w ’y  (5) 

n  n  n- 1 

is  a  sequence  of  mean  zero  orthogonal  random  variables*  Let  v  *  y  -  w  'ye 
where  e  *  (1,1)*  is  the  right  eigenvector  associated  with  1,  and  note  that 


B(c  |  X,  t  k  <  n-1)  -  (Pv)  (X  .)  -  Xv(X  ,)  . 

n  x  n-1  n-1 

Than,  if  x  ia  tha  right  aiganvactor  for  X,  v  “  a^a  +  a^x  for  soma 

«2»  ao  that  y  ■  o^x  (usa  w'v  ■  0  ■  x'Py).  Thus  Pv  »  Xv  and 

x{c  |  X.  t  k  <  n-1)  -  0>  a  saquanca  c  having  this  proparty  ia  cailad  a 
n  k  n 

aartingala  diffaranca  aaquanca,  and  it  follova  from  baaic  propartiaa  of 

conditional  axpactation  that  e  ia  a  aaan  saro  orthogonal  aaquanca,  proving 

n 

(5). 


In  ganaral,  ona  might  hopa  that  for  a  atationary  d  atata  Markov  chain 
Yn  -  w(X  ),  thara  axiat  conatanta  . ,bp,  af ,  *  •  • ,«q,  0  auch  that 


Y  +  bjr  b  Y  -  c  +  a4e  ,+...+  a  e  ♦  0 

n  1  n-1  p  n-p  n  1  n-1  q  n-q 


(6) 


where  tha  en'*  ara  martingala  differences.  Condition  both  aidaa  of  (6)  on 
Xfc,  k  <  n-1,  to  obtain 

(Pp)(X  .)  +  b.Y  -+...+  b  Y  -  a4e  4  +. ..+  a  e  +  0  .  (7) 

n-1  1  n-1  p  n-p  1  n-1  q  n-q 

Prom  (6)  and  (7),  it  follova  that  e  -  p(X  )  -  (Pp)tt  - ) .  By  tha 

n  n  n"  1 

atationarity  of  Yn,  ona  may  back-aolva  in  (€)  to  obtain 


*  "  l  Vn-k  +  B  (8) 

k-0  *  "  * 

whara  -  1.  Subatituting  e  -  u(X  )  -  (Pw)tt  .),  it  follova  from  (8) 
u  n  n  i 

that  (P|i)(X  j)  “  Yib(*n_i)  ia  a  function  of  Xk»  k  <  n-2.  Thin,  in 
ganaral,  holda  only  if  |i  ia  a  conatant  vactor,  and  tharafora  martingala 
diffaranca  rapraaantation a  of  tha  form  (6)  ara  uaually  nonaxiatant  for 
d  >  3. 


5.  Coneluaiona 

Wo  hava  invaatigatad  ARMA  representation  for  white  noiaa  in  a  Markovian 
environment*  Such  rapraaantationa  ara  alwaye  poaaibla,  although  tha  convaraa 
is  inoorract  (not  all  ARMA  procassas  can  ba  raalisad  as  white  noiaa  in  a 
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\ 

i 

Markovian  environment) .  We  hava  also  shorn  that  in  a  special  setting,  the 
noving  avaraga  innovations  are  martingale  differences* 
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